{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "manual-indication",
   "metadata": {},
   "source": [
    "# Chinese Unified Social Credit Codes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "working-price",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "contemporary-house",
   "metadata": {},
   "source": [
    "The function `clean_cn_uscc()` cleans a column containing Chinese Unified Social Credit Code (USCC) strings, and standardizes them in a given format. The function `validate_cn_uscc() ` validates either a single USCC strings, a column of USCC strings or a DataFrame of USCC strings, returning `True` if the value is valid, and `False` otherwise."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "subjective-tuning",
   "metadata": {},
   "source": [
    "USCC strings can be converted to the following formats via the `output_format` parameter:\n",
    "\n",
    "* `compact`: only number strings without any seperators or whitespace, like \"91110000600037341L\"\n",
    "* `standard`: USCC strings with proper whitespace in the proper places. Note that in the case of USCC, the compact format is the same as the standard one.\n",
    "\n",
    "Invalid parsing is handled with the `errors` parameter:\n",
    "\n",
    "* `coerce` (default): invalid parsing will be set to NaN\n",
    "* `ignore`: invalid parsing will return the input\n",
    "* `raise`: invalid parsing will raise an exception\n",
    "\n",
    "The following sections demonstrate the functionality of `clean_cn_uscc()` and `validate_cn_uscc()`. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mature-steps",
   "metadata": {},
   "source": [
    "### An example dataset containing USCC strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "intelligent-hybrid",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"uscc\": [\n",
    "            \"9 1 110000 600037341L\",\n",
    "            \"A1110000600037341L\",\n",
    "            \"51824753556\",\n",
    "            \"51 824 753 556\",\n",
    "            \"hello\",\n",
    "            np.nan,\n",
    "            \"NULL\"\n",
    "        ], \n",
    "        \"address\": [\n",
    "            \"123 Pine Ave.\",\n",
    "            \"main st\",\n",
    "            \"1234 west main heights 57033\",\n",
    "            \"apt 1 789 s maple rd manhattan\",\n",
    "            \"robie house, 789 north main street\",\n",
    "            \"(staples center) 1111 S Figueroa St, Los Angeles\",\n",
    "            \"hello\",\n",
    "        ]\n",
    "    }\n",
    ")\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "competent-banks",
   "metadata": {},
   "source": [
    "## 1. Default `clean_cn_uscc`\n",
    "\n",
    "By default, `clean_cn_uscc` will clean uscc strings and output them in the standard format with proper separators."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "handled-irrigation",
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataprep.clean import clean_cn_uscc\n",
    "clean_cn_uscc(df, column = \"uscc\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "olive-pontiac",
   "metadata": {},
   "source": [
    "## 2. Output formats"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "handy-clark",
   "metadata": {},
   "source": [
    "This section demonstrates the output parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hydraulic-clothing",
   "metadata": {},
   "source": [
    "### `standard` (default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "desperate-thanks",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_cn_uscc(df, column = \"uscc\", output_format=\"standard\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "broadband-glass",
   "metadata": {},
   "source": [
    "### `compact`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "executive-telephone",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_cn_uscc(df, column = \"uscc\", output_format=\"compact\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "miniature-roulette",
   "metadata": {},
   "source": [
    "## 3. `inplace` parameter\n",
    "\n",
    "This deletes the given column from the returned DataFrame. \n",
    "A new column containing cleaned USCC strings is added with a title in the format `\"{original title}_clean\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "biblical-zealand",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_cn_uscc(df, column=\"uscc\", inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sorted-memorial",
   "metadata": {},
   "source": [
    "## 4. `errors` parameter"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "european-fourth",
   "metadata": {},
   "source": [
    "### `coerce` (default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "brown-comment",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_cn_uscc(df, \"uscc\", errors=\"coerce\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fossil-liverpool",
   "metadata": {},
   "source": [
    "### `ignore`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "micro-perspective",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_cn_uscc(df, \"uscc\", errors=\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "welsh-timer",
   "metadata": {},
   "source": [
    "## 4. `validate_cn_uscc()`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fifty-vinyl",
   "metadata": {},
   "source": [
    "`validate_cn_uscc()` returns `True` when the input is a valid USCC. Otherwise it returns `False`.\n",
    "\n",
    "The input of `validate_cn_uscc()` can be a string, a Pandas DataSeries, a Dask DataSeries, a Pandas DataFrame and a dask DataFrame.\n",
    "\n",
    "When the input is a string, a Pandas DataSeries or a Dask DataSeries, user doesn't need to specify a column name to be validated. \n",
    "\n",
    "When the input is a Pandas DataFrame or a dask DataFrame, user can both specify or not specify a column name to be validated. If user specify the column name, `validate_cn_uscc()` only returns the validation result for the specified column. If user doesn't specify the column name, `validate_cn_uscc()` returns the validation result for the whole DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "marked-nowhere",
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataprep.clean import validate_cn_uscc\n",
    "print(validate_cn_uscc(\"9 1 110000 600037341L\"))\n",
    "print(validate_cn_uscc(\"A1110000600037341L\"))\n",
    "print(validate_cn_uscc(\"51824753556\"))\n",
    "print(validate_cn_uscc(\"51 824 753 556\"))\n",
    "print(validate_cn_uscc(\"hello\"))\n",
    "print(validate_cn_uscc(np.nan))\n",
    "print(validate_cn_uscc(\"NULL\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "arbitrary-summer",
   "metadata": {},
   "source": [
    "### Series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "touched-signal",
   "metadata": {},
   "outputs": [],
   "source": [
    "validate_cn_uscc(df[\"uscc\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "naval-complexity",
   "metadata": {},
   "source": [
    "### DataFrame + Specify Column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "growing-postage",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "validate_cn_uscc(df, column=\"uscc\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "medical-cholesterol",
   "metadata": {},
   "source": [
    "### Only DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "given-comparison",
   "metadata": {},
   "outputs": [],
   "source": [
    "validate_cn_uscc(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "informal-vanilla",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
